posted 02-18-2013 08:05 PM         

Next question please.

posted 02-19-2013 08:52 AM            

An example might be on a simple use of drug relevant issue test. If you were to use a drug distribution comparison question you might be asking a question far more relevant to the examinee than simple use.

The same might be the case if you were asking about the sexual molestation or fondling of a child and used a comparsion question that incorporates in its time frame, an older child in the family that he might also have molested at an earlier date.

posted 02-19-2013 11:34 AM         

If you weaken your controls are you setting your examinee up to fail?

Which is better to hang an innocent man or let a guilty man go free?

posted 02-19-2013 01:00 PM            

This is Skip's post and only copied it:

"An example might be on a simple use of drug relevant issue test. If you were to use a drug distribution comparison question you might be asking a question far more relevant to the examinee than simple use.

The same might be the case if you were asking about the sexual molestation or fondling of a child and used a comparsion question that incorporates in its time frame, an older child in the family that he might also have molested at an earlier date."

posted 02-19-2013 04:40 PM            

If the relevant and the comparison are both getting reactions (which is the norm from what I've seen), why assume it's the CQ that is the issue? Maybe it's the RQ? What evidence do you have that it is one over the other?

How do you "weaken" a CQ to allow the examinee to fail properly? After all, you've already decided that's the way it is going to go.

The jury has returned on the inclusive / exclusive comparison questions. Those that are inclusive ("Have you ever stolen anything from anyone?) result in increased accuracy - not a decrease. "Adjusting" the CQs should therefore result in a decrease in accuracy.

In the lab, in theory, every CQ should be too strong. After all they are real transgressions as opposed to the pretend "crime." And yet, we find that's not the case at all. Moreover, some programmed innocent fail. Does that mean asking if they stole the watch, cash, whatever, was too strong? We'd never say that is the case. Why then would we presume the CQs are too strong?

posted 02-19-2013 07:11 PM         

Backster never taught us to "weaken" the controls. The "remedy" was to adjust the time bars and leave the wording of the controls alone.

posted 02-20-2013 09:43 AM            

Backster has changed slightly since 1985 in his teaching because of research. He had the Either/Or rule at one time, and changed to Some and More later regarding reactions on control vs relevant issue questions. The Either/Or rule produced to many inconclusive and incorrect DI calls.

posted 02-23-2013 05:25 PM         

I went through Backster's in 1999. The Either Or and Some and More rules were still being taught. And who could ever forget having to spin the wheel, stand up in front of the class and recite the redemy for correcting "deficient comparrison questions"?

posted 02-23-2013 07:51 PM         

Ted

posted 02-24-2013 11:14 AM            

Stochastic models are good when we have a single objective - like target shooting. Make some holes, adjust the sights, make some more holes, etc., and the process becomes more precise.

It is different when we have two categorical objectives to choose from. In this case adjusting the data one way or another requires that we know in advance which is the correct one, or requires that we permit ourselves to use our personal opinion to influence the result of the test.

It is our goal is to get people to produce truthful or deceptive scores - but it is not our goal or our role to decide in advance to push the scores one direction or another.

Scientific testing involves a simple process: present the stimulus, observe/measure/code the response, lather, rinse, repeat, aggregate the data together, compare the result to the normative data (i.e, cutscores) to determine to determine the level of significance.

Start tweaking on the data collection process and we are going to get a lot of attacks around the notion that we caused the result to come out the way they did.

There seems to be some action underway to impose some order or authority onto the forensic sciences. We can walk away and not play with them, or we can think through the advantages of learning to think carefully about the test in terms of standardized procedures - part of which means determining whether the stimulus should be changed or adjusted during the test administration.

http://www.nist.gov/oles/doj-nist-forensic-science021513.cfm

When the relevant and comparison questions produce equivalent reaction then we would be tempted to assume that the difference in reaction is not significant or that one of the questions is defective. Which one? conservative judgement might say to err on the side of caution and weaken the CQ so to avoid or reduce FN errors. Data tell us that most inconclusive results are actually truthful.

In reality, we cannot know whether the observed lack of difference in one chart means anything or nothing. We cannot determine the signficance of that untill we have all the data. So, adjusting the data requires introducing our human bias into the test. Conservative scientific approach is to collect all the data without changing the stimulus and then determine the level of significance of the result.

No doubt Backster is a genius and we'll use some of his ideas forever. Same with Reid. But if we want to end the drama, then why don't we study the data and then follow the evidence. Strong-side/weak-side/left-side discussions are answerable questions.

If anyone has an interest in helping with this, then please contact myself or Mark Handler. Could be interesting.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 02-24-2013 03:19 PM            

Maybe I have mis-stated what I was trying to say. To clarify, Backster, in 1985 taught that reaction to the relevant question indicated deception to that question, if there control question was showing reaction it was his opinion the control was competing with the relevant issue. This what was taught.

Since that time I have had other instruction from other professionals and don't hold to the Backster premise from 1985. I believe Cleve has even moved away from that hard stance. We were discussing "can comparisons be too strong". Comparisons can compete with the relevant questions when they are not properly formulated and we can get errors. It has everything to do with the manner in which the comparisons are formulated. (pre test)

I do see reaction to controls and relevant in the same examination. Occasionally I see only reaction to the control questions and none to relevant questions. I do not adjust my questions during the actual examination, I have been taught not to do that in many ongoing training seminars.

posted 02-24-2013 04:20 PM            

That is super interesting.

I knew that Baster used to teach that there should be a reaction to either the RQ or the CQ and not both.

It seems the data data don't' support that. If you look at the Ansley survey of scores (Ansley & Krapohl, 1999) they showed it is not the presence or frequency of reaction that differentiates liars from truthtellers but the strength or magnitude of reaction.

Other research has concurred with this - and it goes back to John Reid who had the conscientious wisdom and independent thinking to tell us that we need CQs precisely because it is normal for truthful person to show some reaction to the RQs.

Do they teach a "Some and More" concept now?

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 02-25-2013 09:39 AM            

I was interested in your comments on this. I agree that adjusting the "perceived" strength of a comparison question mid test can only be seen as prejudicial one way or another, I imagine being asked the reason why I dampened down a CQ ? but I can't think of an answer that could not be interpreted as I was trying to Bias the result in some way.

On another point In your comments you said that "Data tell us that most inconclusive results are actually truthful." is there any specific research to back this up! I think it's something a lot of examiners would agree with from a theoretical stance but I would be interested to see it tested.

posted 02-25-2013 06:03 PM            

That's not arbitrary (what you are doing). It's really a form of counterbalancing, which is a generally desired.

Usually you would see it in a study in which half the subjects are exposed to test A before test B and the other half, test B before test A to make sure there is no order effect taking place.

Guy,

posted 02-25-2013 06:13 PM            

It shows the inclusives for the 45 included surveys and experiments.

Unweighted mean of all guilty-inconclusives was 10.0% and 13.7% for innocents.

It's blunt, but the trend is clear.

Maybe Barry can tell if the difference is signfificant. I'm away from home for another week, and my 'puter went up in smoke last week - so I'm reinstalling and recovering a lot of stuff onto a new machine and cannot do it right now.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 02-26-2013 09:33 AM            

I'm convinced "floating windows" in the pneumo tracings and strongest comparison moving against strongest relevant are both "water cooler" science created by bored instructors who discuss polygraph a great deal and run polygraph very little.

Neither procedure really adds anything of value to the process and I am unawaree of any science to support the rationale for doing either.

posted 02-26-2013 10:07 AM            

Something interesting.

Mark and I have been studying the strong-side/weak-side/left-side/right-side.

There does seem to be a slight advantage to the strong side - in terms of test performance using corrected norms for each model. However the difference between the strong-side solution and left-side (preceeding) is so small it is actually meaningless and in all likelihood un-noticable except in occassional annecdotal cases. (and if you look at enough cases you can find something - but that ain't how science works).

So, there is rationale for the Utah solution (preceeding CQ) if the performance is virtually the same and the reliability/simplicity is actually better (less to argue about).

I think this fits with what you are describing.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 02-26-2013 12:24 PM            

quote:

---

You can also look at Appendix B in the meta-analysis.
It shows the inclusives for the 45 included surveys and experiments.

Unweighted mean of all guilty-inconclusives was 10.0% and 13.7% for innocents.

It's blunt, but the trend is clear.

---

I use a continuity correction (because you can't divide by zero, and some studies have 0 INCs), so my numbers are a little different.

I calculated the average Truthful INCs and average Deceptive INCs for the single-issue tests only (including the outliers). (I did a meta-analysis of the reported results in the APA meta-analysis using the number of truthful and deceptive cases, respectively and the number of truthful INCs and deceptive INCs, respectively.) They are different:

Truthful
Average 15.7% (95% CI = 14.6% to 16.9%)

Deceptive:
Average 11.6% (95% CI = 10.7% to 12.6%)

If there were no difference, the confidence intervals would overlap, and you'll notice they don't. So, on average, INCs are more likely to be truthful than deceptive, but it's not a huge difference.

The bottom line is that the INC zone, if you will, protects us from erroneous decisions when a score is (essentially) equally probable for a deceptive or truthful examinee. (It's a good idea.)

posted 02-28-2013 09:44 AM            

It is not quite correct to interpret inconclusive as essentially equally probable for deception or truthtelling. The error statistics for the two categories are unlikely to be exactly the same, and we could actually calculate an odds ratio - and perhaps even conditional probabilities based on these, that the examinee actually belongs to one group or another.

It brings us back to the arbitrariness of hypothesis and significance testing versus the interpretation of p-values and standard errors - but there is always value in rules because they create more consistent and reliable interpretations.

Inconclusive simply means that the numerical scores are not signficant.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)